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Abstract 


Within the past few years, two novel computing techniques, cellular automata and 
neural networks, have shown considerable promise in the solution of problems of a very 
high degree of complexity, such as turbulent fluid How. image processing, and pattern 
recognition. Many of the problems faced in experimental high energy physics are also of this 
nature. Track reconstruction in wire chatabers and cluster finding in cellular calorimeters, 
for instance, involve pattern recognition and high combinatorial complexity since many 
combinations of hits or cells must be considered in order to arrive at the final tracks or 
clusters. Mere we examine in what way connective network methods can be applied to 
some of the problems of experimental high energy physics. [t is found that such problems 
as track and cluster finding adapt naturally to these approaches. When large scale hard- 
wired connective networks become available, it will Бе possible to realize solutions to such 
problems in a fraction of the time required by traditional methods. For certain types of 
problems, faster solutions are already possible using model networks implemented on vector 
or other massively parallel machines. [t should also be possible, using existing technology, 
to build simplified networks that will allow detailed reconstructed event information to be 
used in fast trigger decisions. 


1. Parallel Processing 


The basic tenet of parallel processing, that independent tasks can be performed at 
the same time, is of course not a new idea, but it is only recently, with the ever-decreasing 
cost of computer hardware, that one has dared to think of actuaily putting two or more 
computers to work on the same problem. Parallel processing has made an impact on high 
energy physics in a variety of applications. 


Three parameters can be used to specify the architecture of any parallel processing 
system: 1) the number of processors, 2) tlie power of each processor, and 3) the degree 
of connectivity between processors. In what follows we shall see that difficult problems 
do not necessarily require powerful processors. Depending upon the problem to be solved, 
a large array of simple but tightly coupled processors may be more appropriate than a 
small array of powerful, loosely coupled processors. Processing power of the individual 
processors can be traded off against more processors, or higher connectivity, or both. The 
overall power of a particular architecture is specified not by any one of the parameters, 
but, in some sense, by the product of the three. 


To date, іп high energy physic: , most applications have involved a modest number, а 
few tens, or at most hundreds, of powerful processors which are loosely coupled, i.e., have 


low connectivity. Examples of this approach are the 168/E and 3081/E emulator ‘farms’, 
and the Fermilab ACP system (see ref. (І! fora review of these and other related systems). 
In these applications, individual data events are distributed to the processors, each of which 
completely analyzes an entire event, passes on tlie results, indicates its readiness lo accept 
another event, and son. Beranse the processors independently operate on separate events, 
the amount of cominnnication required between processors is minimal. 


In another ty pe of application, a group of processors work on different parts of the same 
event; one ou the tracking, ane on the calorimetry. and one on the Cherenkov analysis, for 
sophisticaled (smaller individual 


instance, In this case the processors can be somewhat les: 
memory requirement. e.g.) but the degree of connectivity is necessarily larger since the 
results пиві be reassembled at the end of (he calculations to build a complete event or Lo 
таке a trigger decision (the NIKHEF FAMP system is an example of the latter |21). 


Recently, powerful, commercially produced vector compnters have begun to become 
available. These machines are an example of large arrays of very primitive processors 
(each can only add. subtract. multiply. divide. aud perhaps extract square roots) which are 
very highly coupled, Consider for example forming the dot product of two 100-component 
vectors. In a хе ог compnter, each of the 100 multiplications of the individual components 
can be done simultaneously (assuming the maximum machine vector length is at least 
100), but afterwards the results inust all be brought together again to be sumined into the 
dot product. By recognizing those areas of high energy physics data analysis programs 
which contains large numbers of manipalations of independent quantities, aud by recoding 
these sections for application on a vector machine. significant gains in processing speed 
can be had. Examples are the Fermilab E711 and SLAC Mark III vectorized track finding 
programs în which the drift chamber hit pattern is conipared to aa hank of stored templates 
of all possible tracks, 3144; and Ihe CUSD calorimeter clustering algorithm, which uses 
stored neighbor information to group pulse heights together 15). 


2. High Connectivity Parallel Processing 


Two of the most recent developments in parallel processing, and ones to which as 
yet little attention has been given by experimental high energy physicists, are the use of 
cellular automala and neural networks, which are in fact closely related to each other. 
Both consist of arrays of very simple individual processing cells, with multiple inputs and 
outputs, which are connected to each other according to a permanent, fixed pattern which 
depends on the problem to be solved. Thus, these arrays are a bit like a vector machine in 
which the pattern of connections between processors for a given problem has been uniquely 
fixed, being established either in the hardware, or through an imposed addressing scheme. 


Associated with each cell is a variable describing its current output value; it is the set 
ої all output values which ultimately will encode the ‘answer’ to the problem being solved. 
Each cell's output value is uniquely determined [rom the ensemble of its inputs, i.e., the 
ensemble of information received from the cells to which it is connected, according to a 
simple rule which is normally (he same for all cells. Each cell's new output velue is then 
transmitted to the input of cach cell to which it is connected, and the process continues. 


The system is allowed to evolve in this way until a steady state is reached, i.e., until the 
pattern of output values stops changing. These 5ісапу states are called attractors. 


The ‘Game of Life’ of Conway (6] is a popular computer game using cellular automata 
in which the cells are represented by squares in a grid on a display sereen. The player 
assigns random ontpul values to the cells, which are represented by the cell being dark (0), 
or bright (1), and defines a transition rule. Each cell, responding to its neighbors, changes 
its state accarding to the transition rule, making interesting patterns of bright and dark 
dots as the system evolves toward an attractor stale (figure 1 [6)). 


High-connectivity methods are most effective on so-called ‘intractable’ problems in 
which, due to combinatorics, or inherently large numbers of variables, solutions by more 
traditional means would require collossal amounts of computer time. Cellular automata 
are currently being used to study. for example, detailed solutions to turbulence problems 
in Huid mechanics, and the growth of snowflakes |7), two areas long considered too complex 
to be solved in detail by computational methods. Recently, they have even been applied 
to simulations of the distribution of galaxies in the universe [8]. 


One of tlie more famous successes of neural networks is in their application to the Trav- 
eling Salesman Problem, or TSP [9]. lu this problem a list of cities and their coordinates 
is presented, and the computer has to pick the shortest trip that visits each city exactly 
once. The number of possibilities grows as N! so that for more than about 12 cities the 
problem becomes intractable on a traditional, even vector, computer. А neural network of 
AN? neurons, on the other hand, can provide near-optimal solutions in a few network time 
constanis. Time constants of currently available networks are of the order of 1 microsec- 
ond [10]. (These prototype networks are still rather small, but the time constant should 
not be a function of network size.) Another current application of neural networks is the 
NETtalk project developed at Johns Hopkins University [11]. This is a simulated neural 
network wilh some 10,000 interconnections which can be ‘taught’ to translate printed text 
into spoken words by using an iterative 'learning' procedure. 


Neural networks and cellular automata thus are at the opposite end of the parallel 
processing spectrum from, say, emulator farms: the former consist of large numbers of 
simple processors that are heavily interconnected, while the latter consist of moderate 
numbers of powerful processors that are loosely interconnected. 


The following section defines iore precisely tlie properties of the two types of net- 
works; subsequently, most of the discussion will deal with neural networks, with a brief 
return to cellular automata in the section on calorimeter clustering. Associative memory, 
an important property of neural networks, is discussed in section 11. 


3. Characteristics of Cellular Automata and Neural Networks 


As mentioned earlier, cellular antomata and neural networks are closely related, but 
four qualities distinguish a cellular automata system (see [7] for a more detailed treatment). 
First, the output values of the cells are discrete (usually 0 and 1, aithough multistate 
automata are also being studied). Second, each automaton is connected only to a limited 


number of nearby cells, often just its nearest ucighbors. Third, a cellular automata system 
is clocked: with each clock tick, cach autamaton examines its inputs, decides on the basis 
of a transition rule’ whether or not to change ils slate, and sends its new output value 
to the inputs of its neighbors. On the next tick these new inputs are examined, and so 
un. Fourth, the transition rale used, which can have an arbitrary functional form, depends 
upon the problem Го he solved. 


A neural network differs on these four points. First, the output values of the neurons 
are continuous variables, though usually bounded between 0 and I. Second, a given nenron 
can in principle he connected to any or all of the other neurons in the network, including 
d. neural networks are asynchronous, not clocked: 


those which are relatively distant. Tb 
the outpats vary constantly according to an instantaneously evaluated function of the 
inputs. Ir addition, the inputs are integralive over (ime, with an integration time constant 
to be specified. Finally. the function which determines the output froni the inpuls has a 
specific form, and is the same for any problem: the output is a sigmoid function (see fig. 
Only the coefficients in this linear combination 
г) must be specified for a particular problem. 


2) of a linear combination of Ше inputs 
(and of course, the pattern of ronnecliv 


The use of the sigmoid response function is a relatively new idea |91, motivated by 
studies of the response functions of real neurons in the cortex, which have a similar form. 
It was not until the addition of this feature that the calculational properties of neural 
networks were realized; that is, neurons wilh this type of response function seem to exhibit 
computational properties not shared by neurons with simple step-function response (9]. 
The exact shape of the sigmoid curve is not important. but what appear to be the essential 
elements are the existence of a central, nearly linear region, and plateaux as the input 
approaches large positive or negative values. The reasons for the importance of these 
features will be discussed farther in section 5. lt is because of the biological origins of the 
response function, and of course the high degree of interconnectivity, that the mathematical 
networks we study here have come to be called ‘neural’. 


4. Implementation of High-Connectivity Systems 


True hard-wired networks in which the signal processing is done with analog circuitry 
are just beginning to appear. Examples are a 256-neuron, fully connected, silicon-based 
device developed at AT&T Bell Labs [10] and an experimental 10,000 cell optical device, 
developed at Caltech, that uses a hologram to map the output of each cell onto its neigh- 
bors [12]. It shonld he possible, however, with conventional optical and electron-beam 
lithography, to build networks with a connection density of .5- 10° per cm? [10]. Such den- 
sities are possible because the connections between neurons, by far the element required in 
the greatest numbers, can be implemented by simple resistors, which take up considerably 
less area on a silicon chip than a transistor. The basic circuit which needs to be realized 
is shown schemalically in figure 3. This figure shows an array of neurons, represented 
by simple amplifiers, which have normal and inverted outputs in order to supply either 
reinforcing or inhibitory inpul fo other neurons. All of the outputs cross all of the inputs 
in a grid, and resistances, with values proportional ta the desired coefficients, are attached 
at the interstices in order to form the required connection uetwork. The inputs to a given 


neuron are wire-sumuned directly on its input line, The network shown is a fully connected 
one, with the signs of (he connections chosen at randoni. 


As large scale hard-wired net works do not yet exist it is necessary to use modelling to 
study Мейс properties. Because of the structure of a network, these model calculations are 
almost totally veclorizable, and thus can fully exploit the speed of vector machines. Same 
of the fluid flow simnlations have also been done using the the Connection Machine?" 
[13]. This is a commercially produced machine consisting of an array of 65,536 individual 
processors each of which can explicitly address any of the other processors in the array, 
through the intermediary of a Boolean 12-cube routing scheme, This routing scheme is 
necessarily slower than a true hard-wired connection network (which would require billions 
of wires), but is nonetheless very elTicient since eacli word of information sent is never more 
than 12 steps away from its destination. Because of this architecture, the Connection 
Machine is ideal for modelling neural and cellular automata networks. It was designed to 
be able to simulate any size array of pracessurs (even arrays bigger than 65,536 are handled 
transparently), with the number of processors and their interconnections specified by the 


user. 


Et turns ont. perhaps somewhal surprisingly. that for extremely complicated problems, 
even the model of a neural network is fast. faster than a more conventional approach. That 
this is so will be demonstrated in the discussion, in section 6, of the following example. 


5. Example. The Travelling Salesman Problem (TSP) 


We follow here, loosely, the treatment of Hopfield and Tank (9]. Consider, for a trip 
of М cities, ап М by N array of neurons. The horizontal position pf a neuron in the array 
determines the city number, aud the vertical position determines tlie order in which the 
city is visited (see fig. 4). Thus a valid trip is represented hy a different neuron being on in 
each row. Now consider the connections between neurons. Let every neuron be connected 
to each neuron in the rows immediately below and above with a coefficient proportional to 
А — di; where dij is the distance between cities i and 7, and A is a positive constant. That 
is, if neuron i has an output value of Б, it will present a signal proportional to (A — d;;) fi 
at the input of neuron ). The reverse relation is also true since the matrix of coefficients 
is normally taken to be symmetric. Thus, if dij is large, neurons i and ў will tend to 
inhibit each other, with the effect that patterns of neurons with small d;; values between 
neighboring rows are favored. Inhihitive connections, of value -- B, say, are also imposed, 
between each neuron and those in its own row and column. This has the result that if one 
neuron is on in a particular row and column, it will tend to turn off other neurons in the 
same row and column, thus ensuring that valid tours exclusively will be considered. 


Randou starting values are assigued to the neuron outputs, and the system is allowed 
io evolve. Throughout the evolution of the network, each neuron will have an output value 
between D and 1, which is equivalent to saying that the state of the system is described by a 
point in an N-dimensional space, with one dimension for each neuron. [n the final solution, 
each neuron must һауе a value of either 0 or 1, i.e., the solution lies at a corner of an N- 
dimensional hypercube. During the calculation, however, the state of the system moves 


E 


around within the volume of the hypereube, The neurons at this stage are operating in the 
liucar region of their response functiaus. The state of the system during the calculation, 
id tour, can be thought of as a superposition of several 


which does not yet. represent a ve 
possible tours whieh are being considered simultaneously. 


Neurons with step function response lack (his linear region. Simulations have shown 
[9] that the tour minimization ability of ar inferior to that of continuous 
valued neurons, presumably because the ability to simultaneously consider a number of 
tentative solutions is not present ìn thal rase. The interior volume of the calculational 
hypercube is forbidden, aud the system is restricted to jumping from one corner of the 


ich neurons is 


hypercube to another. 


In the TSP, pairs of neurons representing close-together cities reinforce cach other 
through positive feed-back. ie.. a positive change in (he output of one nenron will elicit 
positive changes in the outputs of those neurons with which it has mutually reinforcing 
connections. These neurons will in turn re-stinulate Ше initial neuron, so that, after 
several cycles, the outputs will be pushed into the f = I plateau region of the sigmoid 
response function. Pairs of nenrons whose simullaneous presence adds a large dj; to the 
energy. i.e, those whose connections are predominantly inhibitive, will, similarly, force 
each other to zero alter several cycles. It is in this way that the system evolves to an 
attractor state of neurons with saturated output values of l and 0. 


Hopfield and Tank have shown that the nenral system obeys a set of coupled non-linear 
differential equations, which for the case of a homogeneous network, and in the absence of 
input currents froni outside the network itself, cau be written as: 


des 
Ц Ы а U 
7 = Tutte ~e 


where f is a sigmoid output function, vj is the input value of neuron i, Т.) is the coupling 
strength or coeflicient between neurons i and ĵ, and т is the integration time constant of 
the array. The equations tell us that the change in the input value of a particular neuron 
in a small interval d! is given by two terms, the first being the sum of the output values 
of its neighbors multiplied by their coeflicients, and the second being simply a time decay 


term. 


It can be shown [9),[14] that for such an array, there exists an ‘energy’ (unction E 


given by 


pasa 
EDITION 
¿gd 
which, as long as Т); = Tj, and Т; = 0, will be continuously minimized as the system 
approaches an attractor state. For the TSP, with 7;; — А-а, it is clear that, for valid 


tours, E is minimized when 
valid trip 


y d,, = minimum 
2. 


„i 
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һе. when the total trip length is shortest. * 


The minimum found by the network is not necessarily the global minimum of the 
energy function. In practice, though, it is found that when many minima exist, they are 
ойсп nearly degenerate, as mentioned again in (he next section, and that, where widely 
separated minima do exist, the deepest minima have the widest basins of attraction, thus 
making them the more likely final states of the system. The basin of attraction chosen by 
the system can also be influenced by au appropriate choice of initial state as in the case 
ol associative memory (section LI) (9), 


6. Philosophy of Neural Networks 


А peculiarity of neural networks is that they do not always pick the absolutely shortest 
tour, but rather pick ene of a set of nearly degenerate short tours whose lengths vary 
only slightly from that of the shortest tour. [t ія not yet understood whether this is a 
fundamental limitation of the method or whether it will ultimately be possible to ensure 
perfect performance, but, as Hopfield and Tank point aut, bad solutions in their 30-city test 
network were rejected over good ones by a factor ої 1077. For many applications this kind 
of performance is a good deal more than adequate. On a trip totalling many thousands 
of kilometers, differences of a few kilometers are just not important. Similarly, a very fast 
track finding algorithm which misses а few points or splits a track occasionally is probably 
preferable over a very slow one which is perfect. ln experimental high energy physics, 
perhaps more so than other fields, fast, approximate solutions are at a premium, whether it 
be for triggering purposes, on-line event reconstruclion, or lo provide approximale starting 
points to off-line reconstruction programs. At the same (іце, of course, it is necessary to 
understand possible biases which may be introduced by the use of approximate solutions. 


There are no known ‘fast’ algorilhins for finding optima! solutions to the TSP. Of the 
approximate methods (i.e., those giving near-optimal solutions), model neural networks 
are not at present the best or the fastest. Neural network methods are still of considerable 
interest however, for the TSP as well for other problems, because of their conceptual sim- 
plicity, wide range of applicability, intrinsic parallelism, and promise of eventual hardwire 


implemention. 


It is also possible ta handle intractable’ minimization problems like the TSP using 
the formalism of simulated annealing [16]. In this method, а cost function first is evaluated 
using some starting valnes of the state variables of the system. The state variables are 
then changed by small random amonnts, and the cost function reevaluated. A Metropolis 
algorithm is used to decide whether to accept this new configuration of the system, i.e., if 
the new value of the cost function is smaller than the previous one, the new configuration is 
accepted, but if the new value is larger than the old hy an amount АЕ, it is accepted only 
with probability eAE/T where T'is a control parameter analogous to temperature, This 
method has two advantages: 1) the cost function need not necessarily be expressible as a 


* Much of the mathematies used in describiug neural networks is identical to that found 
in the treatment of a sel of problems called spin glasses [15], named for a type of magnetic 
alloy containing both ferromagnetic and antiferromagnetic spin orderings. 


sum of terms bilinear in the state vaciables, as in the case of neural networks, and 2) the 
system is less likely to become trapped in local minima, since the Metropolis function allows 
the system to escape from them by permitting temporary increases in the cost function, 
In typical applications, an ‘annealing schedule” is followed in which several minimisations 
are made, each time lowering the control parameter Т. Simulated aunealiug has, however, 
two important disadvantages [rom the point of view of high energy physics. The first is 
that the high quality global minima are computationally extremely expensive: simulated 
annealing does not offer the promise of speed that neural networks do 117). The second is 
that there is no apparent way to implement simulated annealing in parallel hardware, 


For problems with high combinatorial complexity, vector computers are faster than 
serial computers because Ihey can ralculate large numbers of combinations simultaneously, 
continuing until all coribinalions have heen exhausted. Neural networks are even faster 
since they effectively employ a directed search in which the solutions are weighted by their 
quality. Bad solutions damp themselves out very quickly through negative feedback. The 
network effectively considers many solutions sumltanennsly via Ihe system ої interconnec- 
tions, continuously evolving toward configurations of lower energy. 


The TSP is an example of a system for which the model of the network, implemented 
on a vector machine, would be faster ап ап exhaustive search. To demonstrate this we 
estimate the number of calculations necessary in the two cases. Consider first a vector 
computer, An М city lour has HN — 1)! possible solulions. Each solution involves the 
summing of А — 1 terms to calculate the tour length. At the eud of this, (N — 1)! 
comparisons must be made to find which combinalion gave the shortest tour. The total 
number of calculations is thus 

РОДИ ~i ЫІ 
(N UF Ww DA 


2 2 2 


NT =(N - 1) 


Now consider the neural network. There are N? nenrons. Each one is connected to 4(N - 1) 
others. In each iteration, each connection must he multiplied by its coefficient, and the 
inputs af each neuron must be summed. This implies 2-4(N — 1) operations per iteration. 
In addition, there will be some 8 or so ather operations per iteration associated with the 


function calculation, etc. Thus, 
Nr = UBA - 1) Яіс» = BON? 


where we have assumed Мас», the number of iterations, is about 10, 2 typical number of 
iterations required for convergence. For a 30 city tour, we find 


32 
NT erhauative search ~ 1+ Ю 


Neural ner v 2+ 10° 


The neural net is 26 orders of magnitude faster, ignoring possible effects due to differences 
in average vector length in the two cases, in spite of the fact that the coefficients have to 


be explicitly multiplied and inputs explicitly summed on each iteration [18]. 
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7. Combinatovics in High Multiplicity Events 


The energy function minimised by a neural network can of course be any function 
expressible as the sum of pairwise terms between the neucons. We consider here a method 
of finding Ше set of n tracks in an event of multiplicity М, supposed large, which has 
the smallest value of some function F ~ У; f(pi pj) where f is any function of the 
four-vectors p; and pj. Suppose, for example, we wish to find the set of л. tracks which 
have the largest overall invariant mass М. First note that 


Ms Ур? є NY peroj 


га izi リー1 


n n 


mi У pp 


зі 18) 


where m; is the mass of the i!" track, We assume that the m; are all roughly equal and that 
thus maximizing the second term above will maximize М? (an alternate method would be 
lo incorporate Ше m, into externally applied bias eurrents to the neurons). 


We construct the М by n neural network shown in figure 5. The horizontal position of 
a neuron indicates the track number, and the vertical position indicates the order of that 
track in the group of n tracks being considered. Valid solutions to the problem have one 
neuron on in each row and column, as in the TSP. To (his end, inhibitive connections are 
made between each neuron and the others in Из row and column, The coefficients between 
two neurons not in the same row or column will be of the form Ti, & p; - pj. The system 
thus will evolve toward the configuration in which the value of М? for the n chosen tracks 
is largest. Events for which this largest value is above some threshold could be flagged for 


further study. 


We can again compare the number of calculations needed to perform ihís task through 
an exhaustive search to that required for a model neural network. We assume that ali the 
Fbi,p,) have been calculated separately in advance. For the exhaustive search, the total 
number of combinations is Nm while the number of sumis to be performed for each 


ed aktoj, 52 ' Я ; ! 
combination is FDE The number of compares to be made at the end is also AP 


so that the total number of calculations is 
М! п! 
Nr. iv = І 
T,ezhaustive search У = n) bres = 2)! ] 
For the neural network, the number of neurons is Мен, the number of connections for each 
neuron also N +n, and the number of calculations per iteration thus roughly 2. N?n?. For 
10 iterations the total number of calculations is thus 


2,2 
NT meural network Y 20. Мл 


The totals for a few particular cases are presented in Table I. For N = 35, n = 4, the 
two methods arc ronghly equivalent, For М = 50, n == 4 the model neural net is better, 


9 


but only by aboul а factor af 2. For N ~ 64, n = 6, an improvement of 10° is predicted, 
and for М = 50, n = 25, an improvement of 10% (although admittedly a resonance with 
25 tracks is not a very sensible thing to want to look for). [t is thus the exact nature of 
the problem which will determine whether а model neural net is faster than an exhaustive 
search; nevertheless, a hardwired network (with all possible tracks precalculated so that 
the Ті; do not have to be changed for each event) should always be fast, for any N and n, 
and could be used, for instance, as part of an experimental trigger (see also (18). 


8. Track Finding With a Neural Network 


Most track finding programs use a considerable amount of computing time in trying 
numerous combinations of points, most of which are immediately rejected, in an attempt 
to find track segments which will Lhen be used to build tracks. Thus, track finding looks 
like a perfect candidate for treatment on a neural network. l'or а complementary review 
of more conventional track finding methods, see the recent article of Н. Grote ПО), 


Consider a set of У space points measured in а TPC. From these points we define a 
set of legal segments, delined as segments joining two points in the set with length less 
than some cutoff. Rc. where Д, has a value of several times the mean distance between 
space points. Thus each point can be thought of is inleracting only with those points 
within radius Ка, to form possible track segments. Each legal segment will be represented 
by a neuron, as in the array in figure 6. The circle in the figure represents the presence 
of the directed segment 3 -- 2. The solution (а the problem will consist of an array in 
which all legal segments which belong to tracks are ‘on’, Each final track will consist 
ol a non-bifurcating, unbroken chain of segments, which can be read out by choosing an 
arbitrary point on the chain and following the sequence in both directions until the ends 
of the track are reached. On a valid track, no point should have more than one directed 
seguen entering or leaving it, and no point should appear more than once. For this 
reason, inhibitive connections are imposed, as in the previous examples, between each 
neuron and those in its row and column. The reinforcing coefficients will be set up such 
that the joining together, end-to-eud, of short segments of similar direction will be favored, 
in order to ensure a smooth track. For the neuron 3 — 2 in the figure, for instance, the 
reinforcing coefficients will be imposed between itself and all the neurons in row 2, i.e., 
neurons 2 4 (ог k = 1, М. This chaining together of segments with similar direction is 
similar in some ways to the minimal spanning tree approach to track finding (20]. 


To see if this approach might actually work, a test network was set up on the VAX 

11/785 at L.A.L. Orsay, ТІ.» reinforcing coefficients were chosen to have the form 
cos" 8; 

Tij x ーーーー 

Tij 

where 0;; is the angle between segments i and j, ту) is the length of the vector sum of 

segments i and j, and n is a small integer. We shall call these type 1 coefficients. The 9 

dependence favors pairs of neurons with similar slopes, while the г dependence assures that 

the shorter neurons arc favored. With (hese coefficients, the lowest energy configurations 

will consist of chains of short segments which follow smooth curves. We can liken the track 
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finding to threading a piece of flexible spring wire through the eyes of a set of fixed needles 
И is the total stored energy in the wire that is being minimised by the network. Clearly 
solutions in which tbe wire bends sharply or doubles back on itself will have higher stored 


energy. 


The track recognition capability that we seek is similar to вопіс of the processes studied 
in early vision research. One of the goals of this field is to understand the mechanisms by 
which visual field data in the retina is processed before being sent to the brain, There. as 
in our case, the basic task js 1o provide algorithins which can extract information about 
objects in the three dimensional environment from sparsely sampled, often noisy, two 
dimensional data fields. The use of computational neural netweras has been investigated 
for such early vision problems as edge «detection, surface reconstruction !21', and velocity 
field estimation i22]. 


The network was tried on a set of simulaled events with from 1 to 6 tracks. The 
sigmoid function used was a piecewise linear approximation as shown in figure 7, and the 
parameters used in these tests were: n = 5, Af = .5r per time step. and Ва = 4.5(r), 
where (rj is the mean distance between adjacent points on the same track. In order to 
keep the computing time and meniory requirements low, the number of points per track 
was kept below 20. Some sample results are shown in figures 3a and 8b, which show 4 
stages in the evolution of а 4 and a 5 Irack event, respectively. In figure 8a the network 
has faultlessly reconstructed a 4 track event with two crossing points. Figure 3b shows a 
somewhat more typical result: confusion in regions where tracks are very close together, 
leading to incorrect ог Шева! (e.g., three neurons at same point) solutions in these regions. 
In the last frame of both figures, evolution has stopped, i.e.. convergence of the system has 
occurred. Convergence usually occurred in less than 10 iterations, і.е., less than 5 time 
constants. It is believed that the overall performance can be improved by a more judicious 
choice of coefficients, as a rigorous optimization has not been done, 


In an attempt to improve the performance for close together tracks, another type of 
coefficient, which we shall call type 2, was tried. These represent connections between 
neurons which do по! share an endpoint, but nevertheless are relatively near to each other. 
The value of the type 2 coefficient between two neurons is proportional to the 1 — x? ofa 
ft to a circle through the four points that determine the two neurons. No type 2 coefficient. 
is assigned if the neurons in question are more than a few В, away from each other, or if 
the radius of the fitted circle is less than several times the length of the longer of the two 
neurons. Because the radius of the fitted circle ìs determined by the neurons themselves, 
only local smoothness is imposed, there is no restriction on the global functional form of 
the resulting path, (although the requirement of a constant radius of curvature is an option 
that could be of interest in triggering, sec sect. 9). The presence of type 2 connections, if 
they prove essential, increases the total number of connections, but not dramatically, since, 
like the type 1 connections, they are local. Wilh both type 1 and type 2 coefficients in 
use, it was possible lo bring two concentric arcs quite close together (e.g., radii of 1.0 and 
.98) without failure. Tests with both type 1 and type 2 coeflicients on higher multiplicity 
events are currently in progress. 


и 


The choice of coefficients seems Го he rather delicate, and a number of attempts 
were made before arriving at forms which gave reasonable results. There is much to be 
learned about how to choose (he form of the coefficients and the relative normalizations 
of reinforcing and inhibitory coefficients. The effect of the gain of the response function, 
the time step per iteration, and the initial conditions an the evolution of the network must 
also be better understood. There will surely be some difficulties to he resolved in passing 
from these simple tests to a full fledged track reconstruction program for a real high energy 
physics experiment, but the results so far are promising. 


Ал interesting feature of this method is that the tracks do not need to come from 
the origin, nor do they need to be helices. The network will find and associate together 
groups of points that follow any reasonably smooth curve al. any location in the volume 
being exanuned. Thus it is ideal (or cases where the magnetic field is not uniform, and for 
finding tracks from secondary vertices. Noise points will in general be far from real tracks, 
and segments including them would make unacceptably large angles with other segments 
in a track. This in turn means that such a configuration is unfavored 'energetically', with 
the (desirable) result that noise points will be simply 'ignored'. 


It would be interesting to determine if track finding with a model neural network is 
fast. To do this properly. a comparison should be made, using realistic data, between a 
model neural program and a ‘conventional’ program. both developed or a vector computer, 
or other massively parallel device, such as the Connection Machine. 'This will be ап 
undertaking of some maguitude, but in the meantime it is possible to make some rough 
approximations. 


First we make two simplifying assumptions. The first is that the coordinates of the 
points are binned, so that points can occur only at discrete locations, In a hardwired 
network it is the granularity of the network which would ensure this characteristic. Па 
given point can have a maximum of m possible neighbors within Яс, and each of these in 
turn m — 1 neighbors, there will then be a set of only m(m — 1) possible different values of 
coefficients for legal segments (for simplicit y, we consider here only type 1 coeficients), and 
this set will be sufficient to describe the entire network. Thus it will not be necessary tu 
calculate new coefĥcients for each event. The second assumption is that, in the model, we 
can ignore points that are r^t ‘on’, thus eliminating a large number of essentially irrelevant 


calculations. 


Let т now be the mean number of ‘on’ space points that are within Е, of any given 
space point. This number will of course depend strongly on the density of measuremeuts 
aloug each track and the spatial density of tracks, bul we interpret m here to be the average 
over the data set chosen. This means that for each of the M points, there are on average 
m legal segments that can connect it to another nearby point. This second point then 
will have m — 1 possible new legal segments that can contain it. Thus the total number 
of reinforcing connections in the network is Nm(m - 1) (since neurons 1 do not share 
an end point are not connected to cach other in the network). A similar argument shows 
that there are roughly 2N rm inhibitory connections for a grand total (10 iterations) of 


2 
Nr model network 7” LON(m* + m). 
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Suppose we choose 'histogramming! as the ‘conventional’ method to which to compare the 
model network. [n one application of this procedure [23] the quantity 


sin(A) = - 


2 


VU Iy S 


is histogrammed for all measured space points. A peak finding program then is applied to 
the resulting histogram. Points that contribute to the same peak in the sin(A) distribution 
will have come frem the same track. To calculate and histogram sin(A) for N points should 
require of the order of ION calculations. The number of calculations in the peak finding 
is proportional lo the number af bins, which we presume to be much smaller than the 
number of points, and thus negligable, although as pointed out in [20], in some cases the 
peak finding can become prohibitively complicated. Thus, 


Хра istogramimning て JON 


This suggests that the histogram method will be considerably faster than the model net- 
work. in those cases where histagramming is feasible. since m will typically be of the order 
of a few tens or so. It must Бе remembered of course, that histogramming will only work 
on helical tracks through the origin, while the ueural approach will work for any kind 
of track. Clearly. a detailed simulation is required to determine which method is better 
overall for a given case. 


9. A Neural Trigger 


Hardware neural networks are capable of providing detailed reconstructed event in- 
formation, on tracks, calorimeter clusters, etc., on a time scale of microseconds, and thus 
are of interest. from the standpoint vf triggering. 


Darbo and Heck ¡24' have developed а contiguity trigger for the Delphi TPC at CERN 
which is similar in many ways to a ueural network. lonization from a track in the TPC 
drifts to the wire planes where it is collected. Avalanches at the wires induce signals on 
the pad arrays behind the wires. These signals give г and Ф information, from pad position 
and sharing, and 9 information, from dritt. time. For each of LU 9 bins, the г, information 
is mapped onto a rectangular array of nodes, called an Image Memory (IM), of dimension 
144 bins in $ and 16 bins in r. Each node in the IM can be connected to its nearest 
neighbors by means of addressable latches. A particular patlern of connections for a given 
node is called a contiguity mask. The contiguity mask pictured at the top of figure 9 can 
be described as follows: each “оп” node is connected to its neighbors above and below, and 
in addition, the right-hand neighbor of each ‘on’ node is also connected to its neighbors 
above and below. Tracks are found by applying the same contiguity mask to all оп” nodes 
and looking for a continuous electrical path across the ІМ. Figure 9 shows an [М which 
contains 2 (racks, one of which (the one on the right) is found using the mask chosen, and 
the other, not found. The efficiency of this trigger can be studied as a function of the type 
of contiguity mask (fig. 10). This trigger, because of its array of simpie nodes and high 
connectivity, rescmbles a neural network. 
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Consider now a “true” neural approach to the same problem. Let each IM node be 
connected to each of its nearest, second-ncarest, and third-nearest neighbors for a total 
of 48 connections per node (figure 11). Following the treatment of the previous section, 
but with М now being the number of possible points (i.e., the number of nodes) and with 
m = 48 being the total number of possible neighbors, not 'on' neighbors, we find 


Nr Мот? +m) 


~ 144 16- (48° + 48) 
Nr 5-108 


for the total number of connections. Based on the circuit densities given in section III. a 
trigger of this design should fit quite handily on a single chip, and be capable of finding 
tracks in an IM on a time scale of microseconds. The exact implementation of the trigger 
needs to be specified before the question of rejection power can be properly adressed, but 
presumably rejection of noise events can be based ou lack of convergence of the network, 
or convergence only ta very shallow minima. The neural net will be somewhat more 
tolerant af chamber inefficiencies than Ше contiguity trigger, since it considers а wider 
range of possible connections. Also, the neural network should be sensitive to tracks of 
any momentum (i.e., curvature), while a given contiguity mask does not (by definition) 
efficiently find tracks whose momenta lie outside the range for which the mask was chosen. 
Thus, the track information is more detailed, which opens the possibility of making more 
sophisticated trigger decisions. 


Ап alternative approach to the neural trigger would be to load in type І and type 
2 coefficients which correspond to tracks of a specified constant radius of curvature. The 
network would then only converge. i.e., give a trigger, when a track with this curvature 
is present. lu this configuration, the loading of the coefficients in the neural trigger is 
analogous to the loading of a contiguity mask in the contiguity trigger. If the tracks are all 
roughly the same length, they will each contribute roughly equally to the energy function, 
so that the energy function could perhaps be used to determine the number of tracks found. 


10. Cluster Finding with a Neural Network or Cellular Automata 


Consider a homogeneous multicell calorimeter in which М cells are above some thresh- 
old. We wisì to partition these N cells into subgroups of contiguons cells, i.e., into clusters 
of energy. Form ап М Бу М array of neurons as in figure 12. In each column, reinforcing 
coefficients are set up between each neuron and those corresponding to its nearest neigh- 
bors in the calorimeter. There are no connections between neurons in different columns, 
nor are there any inhibitory connections. 


As starting values, each nenron on the diagonal is set to 1 and all others are set to 0. 
ln each column, the “оп” neuron will proceed to turn on the neurons corresponding to its 
neighbors in the calorimeter, which in turn start to turn on their neighbors, etc. Finally, 
in each column, all neurons that are in the same clustec as the neuron that was initially 
on in that column will be turned on. The number of clusters will be equal to the nuniber 
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of different kinds of columns, aud the content of each cluster can be read off from any of 
the identical rows associated with that cluster, 


This method was used in a test on the L.A.L. VAX. Four energy clusters, of from 2 
to 10 cells each, were found simultaneously in a 50 by 50 cell calorimeter, With steps of .5 
time constants per iteration, convergence took 3 iterations (though this will clearly depend 
on cluster size). The sigmoid response function used was Ше same as in Ше track finding 
test mentioned earlier. 


The clustering can clearly also be done with cellular automata. In one approach, 
the array of automata is exactly the same as that of the neurons, but instead of using a 
sigmoid response, we use the transition rule that a neuron will turn itself on in a particular 
iteration if at least one of its calorimeter neighbors is on. This system wiil evolve exactly 
as did the neurons. 


These two methods will have roughly the same speed since they are actually almost 
identical. “ In both cases there are М? cells. Let т be the average number of cells to 
which a given cell is connected. There are thus roughly М? -m total connections for either 
of these two methods, and И 15 easily seen that the numbers of calculations per connection 


is roughly one for each case. 


The number of iterations needed can be estimated in the following way. We turn on 
one cell at random. [а the cycle that follows, its nearest neighbors will turn on, in the next 
cycle, its next nearest neighbors, and so on. The cells will begin to turn on in concentric 
‘rings’ which expand away from the chosen cell like ripples in a pond. In each cycle the 
radius of the ring will increase hy roughly one cell, and the expansion continues until the 
edge of the cluster is reached. (For simplicity we have taken the case of an “average” cell 
located near the middle of the cluster.) The number of cycles required is thus proportional 
to the the characteristic linear dimension of the cluster. Assuming a “globular” cluster of 
N; cells, this will give, Мини ~ $V/N;. Clearly the largest cluster will be the determining 
one, so that, 


1 
NTON? cells ~ ¿Nm Nmar 


where ХМ лаг is the number of cells in the biggest cluster. 


A faster cellular automata approach involves using М, multistate automata, with one 
automaton per calorimeter cell (see fig. 13.) Let each automaton's “state variable' be equal 
to the energy contained in thai cell (we could equally well use the cell number instead of 


* A model neural network will always be slower than the automata in this case since 
іп a given iteration, an automaton will turn on all of its neighbors completely, whereas 
the neurons will turn them on only partially, in an amount determined by the response 
function. [п the limit ol a very steep respouse function, the two techniques will be identical. 
A hardwired network, on the other hand, has the potential of being much faster, since 
with a steep enough response function, a given neuron could, in the time of one automaton 
clock tick, begin to turn on its neighbors, its neighbor's neighbors, etc., before reaching 


saturation. 
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the energy). The transition rule this time will be that each automaton takes on as the 
new value of its state variable the largest state variable value of any of its ‘on’ neighbors. 
The system will evolve to a configuration in which all cells belonging to the same cluster 
will have taken on a value equal to the energy of Ihe highest energy cell in that cluster. 
Assuming that по two peak cells had exactly the same energy at the beginning of the 
calculation, this classification will be unique. This approach might well be called the 
‘contagion’ approach: the identity of the unique element in each cluster (the one with 
largest energy. e.g.) isa “germ' which ія passed [rom one automaton to the next until all in 
the cluster have been ‘infected’. The disease cannot spread to other clusters since elements 
in different clusters by definition do not touch cach other. 


With this method, each of the N cells compares its own value to those of its m 
neighbors, for roughly М <m calculations per cycle. The number of iterations will go as 


VN maz as before so that 
- E. - 
NTN rela о ТМ Nus. 


For comparison we consider a more traditional”, vector approach. We form 3, N by 
К arrays. 4, B, and С. all initially set to zero. In each row i of A we set to 1 the i!“ bit, 
as well as the bits corresponding tc :he nearest “оп” neighbors of i. Each row i can now 
be thought of as a vector А; which represents a ‘minicluster’ centered on the cell i. We 
initialize by setting C; = 4; and leave B all zeroes. 

Iu each iteration, we form for each ¿a vector Q, hy or-ing together B; and all vectors 
Aj where j is the index of a non-zero element of С. We then set C; = B; Ө O; and 
finally B; = Qi, where Ф is the exclusive or operator. Thus 8; contains the sum total of 
elements known at this stage to belong the same cluster as i, while Cj contains just the 
new elements of the cluster found in this iteration. The process continues until the С, 
become zero, indicating that all the elements of the clusters have been found. 


For an ‘average’ initial cell near the center of a ‘globular’ cluster, the number of new 
elements tound will increase roughly linearly each iteration, in steps of mi, the mean number 
of neighbors, until the [ast iteration (again determined by the largest cluster) where some 
4YN maz new elements will be found. The number of calculations required in each iteration 
is roughly та per new element so that 


Nr sector ~ Nm|m + 2m + 3m + + 4 Nmaz) 
NT vector ~ BN Nun 


These calculations are rather crude but it appears that, for typical numbers, the N- 
automata approach may be advantageous. 


11. Associative Memory in Neural Networks. 


11.1 Feedback Network 


An exciting aspect of neural networks is their potential for information storage. Sup- 
розе we use an array оГ pixels to represent an image, a face, for instance. We first set up 
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reinforcing coefficients between all the ‘on’ pixels, then extinguish the array. Subsequently, 
if we turn on any set of pixels that were previously on, the system will evolve, because of 
Ше reinforcing coefficients, lo the state in which the image is on once again. The image 
can be thought of as being “stared', and can be ‘recalled’ by specifying a subset of the ‘on’ 
pixels in the image. If we wish to store a second image in this same network, in order 
to ensure ‘noiseless’ recall, it will be necessary to modify the coefficients in the regions of 
overlap of the two images, by including inhibilory coefficients, for instance, so that the 
two images can not turn cach other on. Clearly, the subset of a given image that must 
be specified in order to solicit its recall will have to be larger now: in particular, it must 
contain more than the set of points common to both images, in order to uniquely specify 
the image desired. 


Similarly, as more and more images are added to the “memory”, the coefficients of all 
images stored thus far will have to be modified slightly in order to keep the ‘recollections’ 
distinct. (Also, the subsets of points needed for recall will have to be more specific. This is 
equivalent to saying that the set of points applied as input to the network must be sufficient 
to place the system within the basin of attraction of the target response.) This iterative 
process of learning' in order to store memories was first discussed in detail by Hebb [25]. 
The coefficients are established by the iterative learning procedure: initial values are chosen 
at random, and then Ше coeflicients are adjusted slightly alter each image is presented, 
in such a way as to decrease the difference between the response of the network and the 
desired response. (This diflerence is often expressed as the Hamming distance, defined as 
the number of pixels which are nat identical in the desired and obtained responses.) After 
several iterations through the set of images, the coefficients will be effectively optimised 
for the the set of images chosen. It has been shown (14] that the “Hebbian' coefficients for 
the storage of p memories in an array of N neurons can be expressed as 


ҚР” 
PI 


where Є (= +1) is the value of neuron i in memory configuration н. 


We have here, then, a шеапз of recalling the entirety of a stored pattern merely by 
specifying a unique subset of that pattern. This type of information storage is referred 
lo as associative or content addressable memory. Ít is believed that human (or animal) 
memory of, say, familiar faces or objects, may operate along similar lines, e.g., being able 
to recognize а friend's face even though it is partially obscured. Ап optical device capable 
of carrying out exactly this task is described in (12. In another application, a model 
network was used to identify fingerprints from fragmentary or noisy versions of the prints 


[26]. 


Intuitively, it would seem that at some point the network would become saturated, 
і.е., that the number of coefficients would become insufficient to uniquely specify all the 
memory configurations. For corflicients as defined above, it has been shown [14] that in 
fact perfect or nearly perfect recall of memories is only possible if p < aN, with а ~ .1— 2. 
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In high energy pliysics, the most common form of data storage is the mass storage 
of thousands to millions of events on magnetic kape or in large disk files. The binary 
information contained in a data event can, in principle at least, be represented as a pattern 
of pixels, One can imagine, then, a scenario in which а subset of an event, for example “a 
stiff track, an electromagnetic cluster, and 40 GeV missing energy", is specified, in pixel 
form, to a neural network. The network, if such an event exists, then displays the event. 


In practice, there are a number of problems that will have to be addressed before 
associative memory becomes of use, in this form, al least, in high energy physics, One 
problem has to do with the relationship hetween p and M. To store a data sample of 
1 million events, say, requires (а сх 1) 10 million neurons. The degree of connectivity 
required can not be a priori specified, but in the limiting case of a fully connected network, 
101% connections are required, which seems extremely high. Secondly, each event, in this 
scenario, should apparently be represented by 10 million bits of information, which in 
most high energy physics applications would be quite a bit more than required or desired. 
Finally, in high energy physics, it is usually а subset of events whose characteristics fall 
within certain limits that is desired, rather than a particular event that exactly fulfills 
a given set of criteria. Thus, in order that associative memory be useful in high energy 
physics, и will be necessary to a) arrive at а more appropriate relationship between the 
the number of events, the sizes of the events, and the number of neurons, and b), to find 
а way to express the *answer' as a set of events rather than a single event. 


А number of methods have been put forward which address the first of these require- 
ments. although it is not yet apparent how they might be implemented. One approach 
„is the generalization of the energy function to include trilinear or higher terms. This 
does in fact have the effect of increasing the storage capacity of the network, but the con- 
siderable increase in computational complexity makes this technique unattractive, Other 
approaches center upon the use of dilferent types of learning rules. In one implementation, 
the network js able to continuously store incoming images by ‘forgetting’ the earlier images, 
the storage capacity at any given time being constant 28. The use of non-local learning 
rules can also enhance storage capacity. The ‘Hebbian’ rule is a local one: the coefficient 
between two neurons depends only upon the values of those two neurons in the patterns 
stored. By allowing non-local tenns in the learning rule, values np to а = 2 have been 
achieved 129]. For random patterns, (his represents the maximum storage capacity, but it 
has been shown that arbitrarily large numbers of correlated patterns can be stored, even 
though the total amount of information stored remains limited [30]. Non-local learning 
rules are frowned upon Бу some since they are improbable for biological systems, which 
are after all the model upon which neural networks are supposed Lo be based, but they 
may nonetheless prove to be useful, Another interesting approach to increasing storage 
capacity of neural networks is Ihe use of sparsilication techniques [31]. If z pat.erns of 
т bits are to be stored, a network of пого 2y% -z neurons is sufficient providing that the 
patterns can be encoded into п 0-1 strings of sparsity ~ УТ. 


11.2 Filter Network 


Au associative memory can also be constructed using linear networks without feed- 
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back. The circuit will then be simply figure 3 with the amplifiers and feedback loops 
removed. On the input line i we apply the value, Vj, of pixel i in the input image, which 
we wish to compare to a set of p reference images. We have arcanged that tlie resistance 
J in column i be equal to the reciprocal of €’, the value of pixel i in image j. With this 
prescription, the output. currents in row j are given by V^ , У Є). The output lines thus 
provide a list of the ‘dot products! of the input image with each of the stored reference 
images. Additional circuitry can then be used to pick the image which best matches the 
input (i.e., has the largest dot product). A hardware implementation along these lines 
is described in [32]. In high energy physics, this method might prove useful in template 
matching! applications, such as the track finding methoil of references [3j and [4] mentioned 
earlier, and is worthy of further exploration. 


12. Conclusion 


Pattern recognition has always been an important part of high energy physics data 
analysis. But, as one author has put it (J). “This observational, pattern recognizing task 
. із an activity better malched to the capability of a brain than to that of a computer. 
We have applied the brute force numerical capability of computers to very non-numerical 
problems." We have seen, though, that these types of problems adapt themselves quite 
naturally to treatment with neural networks or cellular automata, which at the same time 
are an extension of the trend toward less intelligen! but more highly connected processors 
in high energy physics that we have remarked upon earlier. 


Encouragiug progress has been made in the study of highly complex phenomena such 
as fluid flow through the use of cellular automata. In high energy physics, some of the 
most complex problems, as we have seen. can also benefit from the use of model neurai 
networks. It has been shown that track finding can be implemented in a connective network 
approach in which segments between measured space points are represented by neurons. 
When a simple conventional approach, such as histogramming. is feasible, model networks, 
due to the high computational overhead that modelling entails, may not offer gains in 
speed, though more rigorous tests will be necessary to be certain of this. The high speed 
of existing harawired networks suggests the investigation of scaled-down, coarse-grained 
networks that could rapidly calculate detailed tracking or calorimeter quantities for use 
in triggering. It is appropriate that these investigations, in addition to further studies 
of model networks, (using, e.g, a Connection Machine) be undertaken in the field of 
experimental high energy physics, traditionally a leader in the development and application 
of state-of-the-art technology and computing techniques. 
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Figure Captions 


1) Four stages in the evolution of a Ganie of Life. The initial state is assigned at random. 
The transition rule states that. in each succeeding step, a cell will take on the value of the 
majority of its four neighbors. If their is no majority, no change takes place. The state 
of the system is shown after 17, 82, and 366 generations, after which no further change 
occurs. Four types of stable or ‘attractor’ configurations, а), b), c), and d) are seen in the 
final frame: these are invariant under the application of the transition rule ( d) is actually 
а 2-state, i.e., invariant under two consecutive applications of the rule). These are simple 
examples of the spontaneous creation of large scale structure by the application of simple 
local rules. 


2) Sigmoid response function. 


3) Schematic of a fully connected network. Neurons, represented as amplifiers, have normal 
and inverted outputs, for making reinforcing and inhibitive connections, respectively. The 
network shown is fally counected, with connections assigned at random. Note the absence 
of connections along the diagonal. 


4) Travelling salesman problem ( TSP). Solid line = reinforcing coefficients A — d;;, broken 
line = inhibitive coefficients —B. Connections for neuron (4,5), represented by ©, are 
shown, other ‘on’ neurons are represented by x. 


5) Combinatoric function minimization network. Connections for neuron (4, 3), represented 
by O, ate shown, other 'on' neurons are represented by x. Solid lines are reinforcing 
coefficients, broken lines are inhibitive coefficients. Each neuron is connected to every 
other neuron. 

6) Neural track finding. a) Connections are shown for neuron 3 一 2, represented by (7). 
The reinforcing coefficients are represented by solid lines, the inhibitive coefficients by 
broken lines. b) The track segment 9 -- 5 ~d ~3-2-6-8 — 1 — T is represented 
in a) by O and the other ‘on’ neurons, x. 


Т) Piecewise linear sigmoid function used in track and cluster finding simulations. 


8a) Track facing with a model neural network. Total network energy, iteration number, 
and total elapsed time, T, are given. Measured space points are represented by crosses, 
neurons by segments joining points, with a circle at the ueuron head indicating direction. 
Only neurons with output values greater than .1 are drawn. In practice, most neurons are 
found to have values near either 0. or 1. This 4 track event is perfectly reconstructed. 


8b) Same notation as 8a). At convergence (final frame), the reconstruction is not perfect. 
Examples of missing or illegal neurons (a), and incorrect choice of neurons (b) are observed. 


9) Using the contiguity mask shown at the top, the trigger finds the track on the right, 
which has a continuous path across the IM, but does not find the track on the left. From 


reference [24]. 


10) Efficiency of Delphi TPC contiguity trigger as a function of p, for 3 contiguity masks. 
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From reference [24]. 

11) ІМ with one node mapped onto Из nearest, second nearest, and third nearest neighbors. 
Each of the 48 arrows constitutes а neuron. Each node has exactly the same pattern of 
connections. Eased on the IM definition of reference 24]. 


12) Cluster finding with N? neurons or cellular automata. The reinforcing connections 
are the same for all columus and are only shawn in column 1. Starting values of 1 are 
put along the diagonal. The hashed arcas indicate cells that will be ‘on’ in the resulting 
attractor. The clusters found are (1,3,4,8) , (2,5,6) , (7,9). 


13) Cluster finding with N multivalued cellular automata. Clusters are the sarae as in the 
example of figure 12. 


Table I. Comparison of approximate total numbers ol calculations necessary to find, among 
а set of N tracks, those n which have the smallest value of a function Е = Suj ftp» pj), 
for an exhaustive search and for a model neural network. 
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